Image data processing method and imaging apparatus

ABSTRACT

An image data processing method for sharpening a captured image; the method defines a first transformation equation to change the pixel values of a defocused image into a focused image using the coordinates of the defocused image and the focused image as arguments, calculates a DetailedFocus function from the optimal solution of the first transformation equation by giving the pixel values of the focused image and the defocused image as educational data, extracts a predetermined number of important points on the DetailedFocus function, defines a second transformation equation to change the pixel values of the important points of a defocused image into a focused image using the coordinates of the defocused image and the focused image as arguments, calculates a SmartFocus function from the second transformation equation, and produces a focused image from a defocused image using the SmartFocus function.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image data processing method and animaging apparatus for sharpening an image that is defocused due to beingcaptured out of focus by an imaging device such as a digital camera.

2. Description of the Related Art

There exist technologies proposed to sharpen a defocused image (alsoknown as blurred image) that are produced when a photograph is capturedout of focus. For example, in Patent Document 1, a method is proposed tocorrect, with high accuracy, images that are heterogeneously blurredboth rotationally and with parallel displacement. This method correctsthe blurring by estimating the blur vector of each pixel of an image andperforming interpolated calculation on each pixel according to the blurvector of the pixel location. The blur vector is estimated using acoordinate transformation matrix for the rotational and horizontalmovement occurred by camera shake.

In Patent Document 2, the blur function (point spread function (PSF)) isset using any one or more of the basic functionality parameters and thecapture parameters which can be used to improve focus of each pixel in a3 dimensional image data procured by capturing the internal structure ofthe photo subject under given capture parameters. And these settings ofthe PSF are used for the blur correction process. From the captured 3dimensional image data, the blurs caused by the basic functionality andcapture conditions are reduced for each pixel by reducing the blurcomponent on the pixel level, and then the image processing using the 3dimensional image data sharpens the image and improves image quality.

In Patent Document 3, in order to sharpen an image that has both focusedareas and out of focus (defocused) areas, all pixels of target image isconsidered to be target pixels, and the estimated blur circle shows theinfluence to the color information of the pixels surrounding the targetpixels. Furthermore, this estimated blur circle is used for the inversePSF, which corrects the image so that the color information of thetarget pixels is not influenced by the surrounding pixels.

Patent Document 1: Japanese Unexamined Patent Application PublicationNo. 2000-298300

Patent Document 2: Japanese Unexamined Patent Application PublicationNo. 2005-58760

Patent Document 3: Japanese Unexamined Patent Application PublicationNo. 2005-63000

SUMMARY OF THE INVENTION

A focus function typically contains negative values, and their sizecompared to corresponding PSF is approximately 10 times larger. Forexample, compared to a 3×3 pixel PSF, the corresponding focus functionwould be about 10×10 pixels in size. Therefore, the processing cost fora focusing process is approximately 10 times larger than the processingcost of the corresponding defocusing process. For this reason, thechallenge is to reduce resource usage such as memory use and CPU load.

When applying the above method to a digital camera with no macro mode,only correcting for the lack of focus will not be sufficient, as thecorrections for the lack of focus and camera shake need to be processedsimultaneously.

The present invention takes the above situation into consideration. Theobject of this invention is to provide an image data processing methodand imaging apparatus that sharpens an image defocused by being capturedout of focus, while reducing CPU load and memory requirements to sharpenan image.

In order to achieve the object, there is provided according to an aspectof the present invention, the image data processing method of thepresent invention utilizes a focus function to produce the focused imagefrom an out of focus image (i.e. defocused image).

The focus function here is defined as a type of filter function. Filterfunctions are linear functions used for such purposes as to blur imagesand to extract edges. For example, the following filter function is usedto blur images.

TABLE 1 0.0625 0.1250 0.0625 0.1250 0.2500 0.1250 0.0625 0.1250 0.0625

If the input image and output image pixel values at coordinate (x, y)are determined as s(x, y) and d(x, y) respectively, image blurringprocessing can be expressed as in the following table.

A defocused image can be produced by using a blur filter such as pointspread function (PSF) on a focused image. This model describes theactual defocused images that are captured. A typical PSF is analogous toa Gaussian function.

The first aspect of the present invention is to, instead of producing ablurred image using PSF, calculate a focus function which performs afocus filter and apply the focus function to a defocused image into afocused image.

Compared to Table 1 above, the focus function will be a filter like theone below.

TABLE 2 −0.125 −0.250 −0.125 −0.250 2.500 −0.250 −0.125 −0.250 −0.125

The unique characteristics of the focus function are, as stated above,the inclusion of negative values, and its larger size when compared tothe corresponding PSF function. For this reason, the present inventionreduces processing cost by altering the focus function.

Described in detail, the image data processing method of the presentinvention produces a focus function based on a focused image and adefocused of the same subject. This focus function is an image dataprocessing method used to sharpen a captured image, the function usesthe coordinates of the defocused image and the coordinates of thefocused image as arguments, and it uses a DetailedFocus function todefine a first transformation equation for pixel values to go from adefocused image to a focused image. The optimal solution for the firsttransformation equation is calculated from the focused image and thedefocused image that act as educational data. Perform an important pointselection process for a predetermined number of important points on theDetailedFocus function which was calculated earlier. After the importantpoints had been extracted in the above important point selectionprocess, the defocused image and the focused image and the SmartFocusfunction are used to define a second transformation equation to changethe pixel values of defocused image into focused image. This secondtransformation equation calculates the SmartFocus function as theoptimal solution from the defocused image and the focused image. Usethis SmartFocus function as the above focus function, so the focusfunction will allow focused image to be produced from a defocused imageobtained from an imaging device.

The present invention allows the sharpening of an image with less CPUload since it does not use an inverse PSF in the image sharpeningprocess. Instead, the DetailedFocus function is calculated in advancefrom a focused image and a defocused image. Then, the focus functionthat concentrates on important points (SmartFocus function) isrecalculated using a method such as least squares or neural networks.The focused image is produced using this recalculated focus function.

This transformation equation 1 includes the following parts:DetailedFocus function values which are Fdetail(i, j), focused imagepixel values which are Ifocus(x, y), defocused image pixel values whichare Idefocus(x+i, y+j), where x and y are integers, and m and n arenatural numbers.

$\begin{matrix}\left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack & \; \\{{I_{focus}\left( {x,y} \right)} = {\sum\limits_{\underset{{- n}<=j<=n}{{- m}<=i<=m}}{{F_{{detail}\;}\left( {i,j} \right)} \cdot {I_{defocus}\left( {{x + i},{y + j}} \right)}}}} & (1)\end{matrix}$

The equation above is used to calculate DetailedFocus functionFdetail(i, j) by applying the least squares method from equation 1 aboveon any coordinate of an image, Ifocus(x, y) and Idefocus(x+i, y+j).

Also, use the following 5 steps to select important points used tocalculate the SmartFocus function.

-   (Step 1) Select the 4 corner values of the DetailedFocus function as    important points.-   (Step 2) Perform Delaunay triangulation on existing important points    that were retained.-   (Step 3) Obtain the predictive focus function values for all points.    Predictive focus function values are obtained by interpolating from    the focus function value of the triangulation apex that the point    belongs to.-   (Step 4) Compare the predictive focus function value and the actual    focus function value for all points, and then add the point with the    greatest difference in value as an important point.-   (Step 5) Repeat Step 2 to Step 4 until the number of important    points is greater than a predetermined value.

After selecting the important points using the important point selectionprocess detailed above, set SmartFocus function values to Fsmart(i, j).If Fsmart(i, j) is an important point then set w=1, if it is not animportant point then set w=0, x and y are integers, and m and n arenatural numbers. Then the transformation equation 2 to change adefocused image to a focused image is defined as follows.

$\begin{matrix}\left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack & \; \\{{I_{focus}\left( {x,y} \right)} = {\sum\limits_{\underset{{- n}<=j<=n}{{- m}<=i<=m}}{w \cdot {F_{{smart}\;}\left( {i,j} \right)} \cdot {I_{defocus}\left( {{x + i},{y + j}} \right)}}}} & (2)\end{matrix}$

Calculate the SmartFocus function Fsmart(i, j) by applying methods suchas least squares or neural networks of above equation 2 to anycoordinate of an image, Ifocus(x, y) and Idefocus(x+i, y+j).

According to another aspect of the present invention, the image dataprocessing method sharpens image data that is defocused and blurred frombeing out of focus and from camera shake. Multiple images are capturedin one shot, and for each image use the SmartFocus function to produceits focused image, and then designate one of the focused images as thereference image. The motion data between this reference image and otherfocused images is calculated, and then this motion data is used tooverlay multiple focused images to produce a blur compensation image.

In the present invention, initially the SmartFocus function is used onthe multiple images captured in rapid succession to produce theirfocused images, and then multi-resolution images for each image isproduced. Then a technique such as those disclosed in InternationalPatent Application WO2006/075394 is used to perform a blur compensationprocess, which sharpens an image that is both out of focus and blurreddue to camera shake.

According to still another aspect of the present invention, the imagingapparatus of the present invention possesses the above SmartFocusfunction, and is equipped with multiple SmartFocus functions fordifferent focal distances. A SmartFocus function is selected for eachimage from a set of SmartFocus functions to produce a focused image. Foreach focused image, select a focus function depending on the edge orcorner values obtained from its pixel values, and then use this focusfunction to produce and output a focused image as the captured image.

According to the present invention, by building in a SmartFocus functioninto an imaging device without a macro function, the imaging device cancapture a focused image without a macro mode. SmartFocus function isimplemented for each focal distance, and this allows it to prioritizeimages depending on a certain predetermined condition such as theiredges and/or corners, and output the most focused image.

According to still another aspect of the present invention, multipleSmartFocus functions correspond to coordinates on an image. A focusedimage is produced by applying SmartFocus functions that correspond tocoordinates on a captured image, and this focused image is output as thecaptured image.

According to the present invention, image distortion by causes such aslens aberration can be corrected with the use of SmartFocus functionsthat correspond to coordinates on an image.

As described above, according to the present invention, focus functionsare calculated in advance, where they are applied to a defocused imagecaptured by an imaging device without macro mode, and they allow sharperimage focus correction without CPU load.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of the digital macro process for the embodimentof the present invention.

FIG. 2 is a functional block diagram of the computing apparatus andimaging apparatus used to perform image data processing for theembodiment of the present invention.

FIG. 3 is an example of a DetailedFocus function obtained using themethod for the first embodiment of the present invention.

FIG. 4 is an example of a SmartFocus function obtained using the methodfor the first embodiment of the present invention.

FIG. 5 is an example of a focus image produced by data processing thatis provided according to an aspect of the present invention where FIG.5A is the original image, FIG. 5B is the defocused image, and FIG. 5C isthe focused image.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments in accordance with this invention will bedescribed below. FIG. 1 is a flowchart of the first embodiment of thedigital macro process.

Firstly, both a focused image and a defocused image are captured asoriginal images, and then both image data are input into computingapparatus 2 such as a personal computer (S101).

An original image focused image and defocused image can be capturedusing the following method.

(Method 1)

A close-up image of a subject, such as a chart, is captured out of focusto obtain a defocused image. Next, to obtain a focused image, the samechart is enlarged and printed out, and then the chart printout iscaptured so that it will be in focus and at the same size as thedefocused image. Then the focused image is input into computingapparatus 2.

(Method 2)

Firstly, an image of a subject is captured by setting an imaging unitwith a macro function to the macro mode. This image is the focusedimage, use this image as an original image, and input this image intocomputing apparatus 2. Next, turn off the macro mode and capture animage of the same subject. This image, which is the defocused imagesince it is not in focus, also input this image into computing apparatus2.

After step S101, the focus function is obtained from the focused imageand the defocused image, and use the function for focused imagecalculation in computing apparatus 2. In this case, the DetailedFocusfunction (S102) is obtained first, and then in order to reduceprocessing costs of the CPU, the SmartFocus function is produced onlyretaining important points (S103). The algorithm to determine importantpoints is not specified here.

This SmartFocus function is implemented onto a camera (imagingapparatus) 1, without macro mode. Focused image is produced from adefocused image captured using this camera (S104).

FIG. 2 is a functional block diagram of computing apparatus 2 andimaging apparatus 1 used to perform the above process. Computingapparatus 2 includes input device 13 to input external data, outputdevice 13 to output data, processing device 10 to perform computationalprocesses, and memory device 11 to store data. For input device 12 andoutput device 13, USB, FDD, CD/DVD drive device, LAN ports arepossibilities. Although not in the diagram, the apparatus is capable ofinterfacing with devices such as keyboard, mouse, display, and printer.

Processing device 10 includes image data input unit 21 to input imagedata and store it in memory device 11, DetailedFocus functioncomputation unit 22 to derive DetailedFocus function from an input imagedata, SmartFocus function computation unit 23 to derive SmartFocusfunction from DetailedFocus function, and output unit 24 to output datafrom memory device 11 through output device 13. Each unit from 21 to 24can be implemented as a computer program.

Imaging apparatus 1 includes imaging unit 4 without a macro function,and focused image production unit 42 which computes focused image data43 from implemented SmartFocus function data 35 and defocused image data41 which was captured using imaging unit 4.

Next, operations of apparatus 1 and apparatus 2 are described.

First, a method of producing SmartFocus function data using computingapparatus 2 is described here.

<Image Data Inputting Process>

A focused image 31 and a defocused image 32 of the subject are capturedusing one of the methods stated in the above step S101, and thesecaptured images are input into memory device 11 through input device 12and image data input unit 21 of computing apparatus 2.

After the input of both image 31 and image 32, the DetailedFocusfunction computation unit 22 is activated to perform the followingprocess. Computation method for DetailedFocus function is describedbelow.

<DetailedFocus Function Computation Process>

DetailedFocus function is obtained after an original image and adefocused image are aligned. Here, a focus function is a filter functionwith relatively large pixel size. DetailedFocus function is a focusfunction with function values based on a unit of one pixel (or severalpixels), this improves accuracy but the increase in data volume alsoincreases the required time for focus computation processing.

Equation 1 above is formulated when DetailedFocus function values areFdetail(i, j), original image pixel values are Ifocus(x, y), anddefocused image pixel values are Idefocus(x+i, y+j).

Then, DetailedFocus function Fdetail(i, j) is calculated by applying theleast squares method to any coordinate of an image Ifocus(x, y) andIdefocus(x+i, y+j). This DetailedFocus function is computed usingDetailedFocus function computation unit 22.

FIG. 3 shows a DetailedFocus function obtained using this method.Concentric patterns of small blocks can be seen here, which shows thatvalues fluctuate between positive and negative in concentric circles.

There need not be DetailedFocus function for every single pixel, asusing a selected sampling step such as 2 pixels or 3 pixels units ispossible. Usually, when the number of pixels is increased in a focusfunction, the computation time increases as well, so the sampling stepis adjusted to shorten the computation time.

The computational time required to obtain detailed filter function (i.e.DetailedFocus function) values for width Nw and height Nh is O(Nw²*Nh²).Therefore the amount of computation time required becomes 1/16^(th) justby changing the sampling step from 1 pixel to 2 pixels. However, as thesampling step increases in size, the accuracy decreases. FIG. 3 is theresult of the detailed filter function value of 27 by 27 and samplingstep of 2, white and grey squares in the figure indicates positivevalues, the x's indicate negative values, and lighter colors indicategreater absolute values. The pixel size is 53 by 53 pixels for thisfilter function image.

With detailed filter function values of Nw*Nh values, pixel step s,pixel size Sw*Sh, then the following equations are true.Sw=s*(Nw−1)+1Sh=s*(Nh−1)+1<Computation Process for SmartFocus Function>

Next, SmartFocus function computation unit 23 is activated, and aSmartFocus function is obtained from a DetailedFocus function. ADetailedFocus function has a large number of points, therefore itrequires a fair amount of time to compute and produce a focused image.

Therefore, points that are not important in the DetailedFocus functiondo not retain their value, only values of important points will beretained to be used in focused image calculation. This focus functiononly using important points is the SmartFocus function.

A user will input a DetailedFocus function, number of important pointsNimp, and original image/defocused image to calculate the SmartFocusfunction. Nimp is 0 or greater and less than or equal to Nw*Nh. As thevalue of Nimp increases the SmartFocus function becomes more accurate,but more time consuming to focus an image.

Firstly, the SmartFocus function computation unit 23 selects importantpoints using the following steps.

-   (S1) Select the 4 corner values of the DetailedFocus function as    important points.-   (S2) Perform Delaunay triangulation on existing important points    that were retained.-   (S3) Obtain the predictive focus function values for all points.    Predictive focus function values are obtained by interpolating from    the focus function value of the triangle apex that the point belongs    to.-   (S4) Compare the predictive focus function value and the actual    focus function value for all points, and add the point with the    largest difference in value as an important point.-   (S5) Repeat (S2) to (S4) until the number of important points, Nimp,    is greater than a predetermined value.

One of characteristics of the SmartFocus function computation process ofthis embodiment is that SmartFocus function computation unit 23 selectsimportant points, then after Step S5, the focus function Fsmart(i, j) ofthe important points is recalculated using the least squares method. Thetransformation equation to a defocused image is formulated as above inequation 2.

The focus image accuracy is improved by not using DetailedFocus functionvalues Fdetail(i, j) of the important points, and recalculating focusfunction value Fsmart(i, j) instead.

FIG. 4 is an example where the SmartFocus function was obtained throughthe above process, the DetailedFocus function initially had 729independent values, but its computed SmartFocus function only has 256independent values. In FIG. 3 and FIG. 4, all black boxes with nosquares or x's indicate that no value is contained.

<Implementation and Operation in an Imaging Apparatus>

The SmartFocus function computation unit 23 produces SmartFocus functiondata, which is output through output unit 24 and output device 13, andthen implemented onto the memory device of an imaging apparatus 1.

And then an image is captured by imaging unit 4 without macro function,which is a defocused image, and then the SmartFocus function data andequation 2 is used to produce a focused image data in focused imageproduction unit 42 in imaging apparatus 1.

FIG. 5 is one example of a focused image produced by data processingthat is provided according to an aspect of the present invention of thisembodiment. FIG. 5A is the original image used to compute the focusedfunction, FIG. 5B is the defocused image, and FIG. 5C is the focusedimage which was made sharper by the focus function.

One possible solution would have prepared several patterns of focusfunctions that apply to different focal distances, and then use the edgestrength (sharpness) of each focused image that was calculated usingeach focus function and have them compared to each other to select thebest one.

Yet another possible method would have different focus functionsprepared according to coordinates on an image (including areas) such asdistance from the center of image, and apply different focus functionsdepending on the coordinate on an image to produce a focused image. Thiswill have the effect of being able to correct image distortions causedby lens aberrations.

According to this embodiment of the present invention, a focused imagecan be produced without burdening the CPU with heavy load while alsousing little memory due to the use of SmartFocus functions which will becomputed and built into an imaging apparatus to produce a focused image.

A second embodiment of the present invention is described here. Thisembodiment describes the procedure for correcting and sharpening animage that is both out of focus and has camera shake when it wascaptured by imaging unit 4 without a macro function.

For an image where it is both out of focus and has camera shake, thefollowing steps S11 to S15 can be taken to produce a focused imagecorrected for camera shake.

-   (S11) First, produce an intensity image of n defocused images that    were captured in rapid succession.-   (S12) Next, extract feature points from each intensity image.-   (S13) Of the n intensity images, use one of its intensity images as    the reference image, such as the first one which allows images    obtained afterwards to become target images, and by performing the    feature point matching process, the motion data can be calculated.    Therefore, calculate the motion data between the corresponding    feature points of the reference image and each of n−1 target images.-   (S14) Use a SmartFocus function to produce focused images of the n    defocused images, and for each focused image perform the intensity    image production process of step S11 and feature point extraction    process of step S12.-   (S15) Calculate the motion data of a focused image using a motion    data of a defocused image obtained in step S13 as its initial value,    and this motion data is applied to all intensity images of the n    focused images produced in step S14. As the motion data is    calculated, continuously apply motion correction and merge the image    data with the reference image.

According to this embodiment of the present invention, feature pointsare first extracted from defocused images, then motion data is obtainedafterwards, therefore the number of levels of the multi-resolution imageis decreased, or multi-resolution image processing is avoided altogetherby selecting parts with strong features as feature points. This reducesthe possible mismatching between feature points and also reduces CPUload. Also, by using the motion data obtained from a defocused image asits initial motion data for a lower resolution focused image, camerashake correction process and image focusing process are combined into asingle process, thus decreasing execution time while realizing highlyaccurate camera shake correction and focused image production.

What is claimed is:
 1. An image data processing method for sharpening acaptured image using a focus function produced based on a focused imageand a defocused image of a subject, comprising: defining a firsttransformation equation that includes a DetailedFocus function, tochange the pixel values of a defocused image into the pixel values of afocused image using the coordinates of the defocused image and thecoordinates of the focused image as arguments; calculating aDetailedFocus function as the optimal solution for the firsttransformation equation by using the pixel values of the focused imageand the defocused image as educational data; performing an importantpoint selection process for extracting a predetermined number ofimportant points on the DetailedFocus function; defining a secondtransformation equation that includes a SmartFocus function, to changethe pixel values of a defocused image into the pixel values of a focusedimage based on the important points using the coordinates of thedefocused image and the coordinates of the focused image as arguments,after the important point selection process; calculating a SmartFocusfunction as the optimal solution for the second transformation equationby using the pixel values of the focused image and the defocused imageas educational data; and producing a focused image from a defocusedimage captured by an imaging device using the SmartFocus function assaid focus function.
 2. The image data processing method according toclaim 1, wherein the first transformation equation is described as$\begin{matrix}{{I_{focus}\left( {x,y} \right)} = {\sum\limits_{\underset{{- n}<=j<=n}{{- m}<=i<=m}}{{F_{{detail}\;}\left( {i,j} \right)} \cdot {I_{defocus}\left( {{x + i},{y + j}} \right)}}}} & \;\end{matrix}$ where Fdetail(i, j) is values of the DetailedFocusfunction, Ifocus(x, y) is focused image pixel values, Idefocus(x+i, y+j)is defocused image pixel values, x and y are integers, and m and n arenatural numbers.
 3. The image data processing method according to claim1, wherein the important point selection process comprises the followingsteps: (Step 1) selecting 4 corners of the DetailedFocus function asimportant points; (Step 2) performing Delaunay triangulation on existingimportant points; (Step 3) calculating the predictive focus functionvalues for all points by interpolating from the focus function values ofthe triangulation apex that the point belongs to; (Step 4) comparing thepredictive focus function value and the actual focus function value forall points, and then add the point with the greatest difference in valueas an important point; and (Step 5) repeating Step 2 to Step 4 until thenumber of important points is greater than a predetermined value.
 4. Theimage data processing method according to claim 1, wherein the secondtransformation equation is described as${I_{focus}\left( {x,y} \right)} = {\sum\limits_{\underset{{- n}<=j<=n}{{- m}<=i<=m}}{w \cdot {F_{{smart}\;}\left( {i,j} \right)} \cdot {I_{defocus}\left( {{x + i},{y + j}} \right)}}}$where Fsmart(i, j) is values of the SmartFocus function, If Fsmart(i, j)is an important point then set w=1, if Fsmart(i, j) is not an importantpoint then set w=0, x and y are integers, and m and n are naturalnumbers.
 5. The image data processing method for sharpening image datathat is defocused and blurred from being out of focus and from camerashake, comprising: capturing multiple images in one shot; producing afocused image of each of the multiple images using any one of image dataprocessing methods according to claim 1; and thereafter designating oneof the focused images as a reference image and calculating the motiondata between the reference image and each of other focused images; andproducing a blur compensation image by overlaying the reference imageand other focused images using the motion data.
 6. The imaging apparatuswhich possesses the SmartFocus function calculated based on any one ofthe image data processing methods according to claims 1 to 5,comprising: a memory device for storing multiple SmartFocus functionsfor each different focal distance; a imaging means for capturing animage; and a focused image production means for producing focused imagesusing each of the multiple SmartFocus functions to the captured image,selecting one SmartFocus function among the multiple SmartFocusfunctions depending on the edge or corner values obtained from pixelvalues of the focused images, and outputting the focused image producedby the selected SmartFocus function as the captured image.
 7. Theimaging apparatus which possesses the SmartFocus function calculatedbased on any one of the image data processing methods according toclaims 1 to 5, comprising: a memory device for storing multipleSmartFocus functions corresponding to coordinates on an image; animaging means for capturing an image; and a focused image productionmeans for producing a focused image by applying one or more SmartFocusfunctions that correspond to coordinates on the captured image, andoutputting the focused image as the captured image.